Fitness action recognition is the core of the intelligent fitness system. In order to improve the accuracy and speed of fitness action recognition algorithm, and reduce the influence of the global displacement of fitness actions on the recognition results, a fitness action recognition method based on human skeleton feature encoding was proposed which included three steps:firstly, the simplified human skeleton model was constructed, and the information of skeleton model's joint point coordinates was extracted through the human pose estimation technology; secondly, the action feature region was extracted by using the human central projection method in order to eliminate the influence of the global displacement on action recognition; finally, the feature region was encoded as the feature vector and input to a multi-classifier to realize the action recognition, at the same time the length of the feature vector was optimized for improving the recognition rate and speed. Experiment results showed that the proposed method achieved the recognition rate of 97.24% on the self-built fitness dataset with 28 types of fitness actions, which verified the effectiveness of this method to recognize different types of fitness actions; on the public KTH and Weizmann datasets, the recognition rates of the proposed method were 91.67% and 90% respectively, higher than those of other similar methods.